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INTRODUCTION 


Each year, students, families, schools, and other taxpayers collectively invest over half a trillion dollars in U.S. 


higher education.' For most students wh 


o complete a quality credential or degree, higher education proves 


a powerful lever of upward mobility.2 However, as more students have had to borrow, and borrow more, to 
even attempt a degree or credential program, some are ending up even worse off than before they started, 
with thousands of dollars of debt and little or no increased earnings power to pay it off. As of June 2018, a 
wers were in default on roughly $159 billion dollars in federal student loans.? 


record 8.9 million borro 


4+ 


Given the magnitude of the investment in higher education, and the wide range of outcomes students ex- 


perience, there is a clea 


r need for comparable, accurate information on the outcomes of differen 


While the value of college cannot be reduced solel 
for students who consistently rank the ability to get a good job as among the most important factors when 
deciding whether and where to attend college, and crucial for policymakers who are increasingly concerned 


about labor market outcomes associated with higher education.’ 


4 


4 


t programs. 
y to economic outcomes, this information is important 


Unfortunately, clear and accurate information on employment outcomes is not currently available. Instead, 


+ f 


students and other stakeholders are offered an array of employment metrics calculated by different sub- 
grams at the behest of states, accreditors, and the federal government, and using a 


sets of colleges and pro 


variety of definitions. The resulting pa 


+ 


tchwork of u 


ncoordinated data makes meaningful comparison across 


programs and colleges near impossible and leaves major questions about the accuracy and reliability of 
the available information. For example, one college in Texas reports that its vocational nursing certificate 


program has a job placement rate of 62 percent based on the state definition and 33 percent 


accreditor’'s definition.® 


4 


based on its 


Existing employment metrics are also difficult to verify, which enables misleading or intentionally deceptive 
information being provided to students. Federal investigations found that Corinthian Colleges misled tens 

of thousands of students by systematically misrepresenting programs’ job placement rates and that Career 
Education Corporation paid $10 million to settle claims that it manipulated and falsified job placement data.® 
Finally, even when metrics are clear and accurate, students and others may have difficulties accessing them. 
The adoption of standardized, verifiable, and accessible employment metrics is a critical step that would 
improve decision-making. 


Visualized in Figure 1, e 
widely known as “job p 


for which they were trained, as well as additional measures such as “threshold earnings rates” tha 


mployment metrics are a subset of student success metrics that include what are 
acement rates,” which measure the share of graduates employed in the occupations 


+ 


Lt Mea- 


sure the share of a program's graduates who are employed and earning above a target amount on a yearly 
abor market metrics—such as median earnings—job placement rates and threshold 
earnings rates can be particularly valuable because they capture not just the amount successful graduates 
are earning, but also the share of students who are meeting a basic benchmark of success. As a supplement 


basis. Unlike other key 


4 


to other critical student success and labor market 


programs that increase 
low-paying but otherwi 


students’ upward mobility, 


metrics, these two employment metrics can help identify 
while not penalizing programs that prepare students for 


se rewarding or important 


jobs. 
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METRICS OF STUDENT SUCCESS 


_ TT 


OTHER METRICS (E.G. 
LABOR MARKET METRICS LEARNING, PERSISTENCE, 


COMPLETION) 


OTHER LABOR MARKET METRICS 
(E.G. MEDIAN EARNINGS, 
EMPLOYMENT METRICS RETURN ON INVESTMENT, 
LOAN REPAYMENT) 


THRESHOLD EARNINGS RATE JOB PLACEMENT RATE 


OTHER EMPLOYMENT 
METRICS (E.G. THE 
OVERALL SHARE OF 
GRADUATES EMPLOYED) 


THE SHARE OF GRADUATES | THE SHARE OF GRADUATES EM- 
EMPLOYED AND EARNING PLOYED IN OCCUPATIONS FOR 
ABOVE A CERTAIN AMOUNT | WHICH THEY WERE TRAINED 


This report details existing requirements that govern the calculation and provision of employment metrics 
for each of the three entities tasked with oversight of the U.S. higher education system: accrediting agen- 
cies, state governments, and the federal government. The report documents challenges to comparability, 
accuracy, and accessibilty where employment metrics are currently provided, and recommends federal and 
state level policy changes that must be made for job placement and threshold earnings metrics to live up to 
their potential to provide students with useful information. 


Specifically, to ensure that job placement rates are consistent, accurate, and transparent enough to inform 
decision-making, we recommend that the federal government take the lead in standardizing the job place- 
ment rate definitions used by federal, state, and accrediting agencies to evaluate programs that prepare 
students for gainful employment. We also recommend that new investments be made in state databases 
that, like those already established in leading states, can be used to verify job placement rates as well as 
other important employment metrics. 


improve the availability and utility of threshold earnings rates, we recommend that the Department of 
ucation immediately restore to the College Scorecard tool the threshold earnings rate metric that is cur- 
rently calculated using a federal data match. We also recommend improving on that metric by calculating 
d publishing threshold earnings rates at the program level, in addition to the school level. 


It is also imperative that Congress authorize the creation of a federal student-level data network (SLDN) to 
enable the calculation of federal employment metrics that include all students attending higher education 
institutions. 


4 
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More specifically, we recommend that: 


1. Federal, state, and accrediting agencies take steps to create standardized, verifiable job placement rates 


for career education programs by: 


a. Adopting a specific, standardized definition of job placement for use by federal, state, and accredit- 


ing agencies; 


b. Building state data system capabilities to allow the calculation and verification of job placement 


rates through state data matches; and 


c. Maintaining and improving job placement rate disclosures. 


2. The federal government create and publish a nationally comparable, progra 
rate through a federal data match by: 


m-level threshold earnings 


a. Restoring important information regarding the wages earned by graduates above $28,000 to the 


public facing College Scorecard tool; 


b. Reporting College Scorecard employment metrics, including threshold earnings rates, by program; 


and 


c. Authorizing a federal student-level data network to improve the calcu 
metrics. 


Ultimately, employment metrics have the potential to help students, families, 
stakeholders make better decisions about where and when to invest time and 


4 


improve these important tools. 


CURRENT EMPLOYMENT METRICS CANNOT BE COMPARED 


While students and others may want information on their employment prospects, the current system 


simply does not provide it. An uncoordinated array of federal, state, and accre 


ation of federal employment 


policymakers, and other 
higher education dollars. But 


in order to achieve this, they must have a meaningful and consistent definition, be collected and verified in 
a trustworthy manner, and be accessible to students and other decision-makers. This report further details 
how current employment metrics fail to meet each of these criteria and why it is critical to take steps to 


diting agency standards leads 


to inconsistently defined employment metrics across institutions and programs, making meaningful insight 


and comparison-shopping difficult or even impossible. 


+ 


Students must be able 


to clearly interpret and compare employment metrics to usefully act upon them. Re- 


search finds that attention is a scarce resource, and that there are serious limitations to individuals’ informa- 


tion bandwidths. Therefore, simple and standardized information is the most | 


+ 


ikely to be effective.’ 


Job placement rates provide a textbook example of how the wide variation in definitions and calculations 


render the rates almost impossible to compare across schools. Calculations of how many of a school 


I 


Ss grad- 


uates have obtained employment in their field of study vary widely. Factors that can be inconsistent include: 
what to count as in-field placement (some calculations count related fields while others do not); how much 


a former student must work (some calculations count graduates working just 


a few hours per week while 


others require full-time hours); the length of time graduates must remain employed to be counted as placed 


(some job placement calculations will count employment as short as one day, 
tent employment for at least several weeks or months); how much a graduate 


while others require consis- 
must work (some calcula- 


tions allow graduates working just a few hours a week to be counted as placed in the field); the timing of 
former students’ employment (some rates require students to be employed within six months of graduating 
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while others update their rates for years after students finish the program). Other important assumptions 


4 


driving a job placement calculation include how to count students who already had a job in the field before 
entering the program or students who are gainfully employed but not in their field of study. 


Depending on its location and type, a particular postsecondary program may be required by one or more 
overseers to calculate no employment metrics at all, calculate a single metric, or calculate multiple metrics. 
This patchwork of incomparable data stems from the inconsistent approaches to whether and how to cal- 
culate employment metrics required by accreditors, state regulators, and the federal government—together 


referred to as 


the “triad” —that oversees higher education. 


Accrediting Agencies 


Accrediting a 


£ 


quality of coll 


+ 


sions and set 


The Higher Education Act (HEA), which was last reauthorized in 2008, requires that institutions obtain 


accreditation 


gencies serve as gatekeepers to federal student aid dollars by assessing and approving the 
eges and universities. Each agency oversees a different array of institutions with varying mis- 
s its own standards, leading to a wide array of employment metric requirements. 


from a federally recognized accrediting agency to access federal student aid dollars, the largest 


source of federal higher education funding. Among other mandates, these nongovernmental agencies must 


evaluate “success with respect to student achievement in relation to the institution's mission...including, 


4+ 


as appropriat 


e, consideration of State licensing examinations, consideration of course completion, and job 


placement rates.”® Agencies must either take adverse action against any program that does not meet ac- 


+ 


creditation st 
or the progra 


these seven r 
from liberal a 
7 


report job pla 
ment metrics 


+ 


nstitutions a 
career and te 
ated nationa 


accreditors have repea 


and multiple 
placement ra 
all of 
calcula 


4 


ting th 


While it is at 


tion. Thus, for the vast majority o 
the job placement rates at particular colleges. 


andards or require that the program bring itself into compliance within the shorter of two years 
m’'s length. 


ost degree-granting institutions acquire accreditation through one of seven regional accrediting bodies, 
which cover a wide range of institutions in a particular geographic area.'"° As can be seen in Table 1, six of 


4 


egional accreditors that oversee all non-profit and public colleges and universities, ranging 

rts colleges to community colleges, do not require colleges to track employment outcomes or 
cement rates. Regional accreditors do not issue standards regarding how to calculate employ- 
nor do they set specific benchmarks. While four of the regional accreditors list employment 


+ 


metrics as a possible way for institutions to demonstrate student success, they do not require such informa- 


£ 


students attending degree granting programs, there is no way to compare 


nd programs that primarily grant career-focused degrees or non-degree credentials—such as 


4 


chnical education certificates—typically acquire accreditation through one of seven career-re- 
accreditors. It is important to note that these requirements are not always enforced. National 


+ 


tedly been accused of failing to adequately audit reporting of job placement rates 


institutions have also been forced to settle lawsuits related to misleading or fraudulent job 


4 


tes, as further discussed in the next section. Table 1 lists the five largest of these accreditors, 


which require programs they oversee to report job placement rates, outline a specific methodology for 


ese rates, and hold programs to certain benchmarks. 


4 


least hypothetically possible to compare job placement rates between institutions or programs 


with the same national accreditor, even that comparison requires the student to understand if a particular 
school or program has the same accreditor or program. Moreover, as detailed in Table 2, different national 
accreditors define job placement rates so differently that meaningful comparisons across accreditors are 
virtually impossible. 
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TABLE 1: EMPLOYMENT METRIC REQUIREMENTS ACROSS ACCREDITATION AGENCIES 


Employment Metrics 


Accrediting Agency Number of Number of Public 
Institutions* Undergrads* Reporting Required? Disclosure Relevant Standards Benchmark 
Required? 


Standard 4.A.6: The institution evaluates the success of its graduates. The institution assures that the degree or certificate programs it 

represents as preparation for advanced study or employment accomplish these purposes. For all programs, the institution looks to indicators Set by institu- 
it deems appropriate to its mission, such as employment rates, admission rates to advanced degree programs, and participation rates in fellow- | tions 

ships, internships, and special programs (e.g., Peace Corps and Americorps). 


No, but listed as a potential 
post-completion success 
metric 


Degree-granting institutions in AK, AZ, 
CO, IL, IN, 10, KS, MI, MN, MO, NE, NM, 
ND, OH, OK, SD, WV, WI, and WY 


Higher Learning Commission (HLC)" 4,364,893 


a 
° 


Standard IV.1.d: An accredited institution possesses and demonstrates the following attributes or activities...processes designed to enhance 
the successful achievement of students’ educational goals including certificate and degree completion, transfer to other institutions, and 
post-completion placement. 


No, but listed as a potential 


Degree-granting institutions in DE, DC, 
MD, NJ, NY, PA, Puerto Rico, and the U. S. 
Virgin Islands 


Set by institu- 
tions 


Middle States Commission on Higher 


Education (MSCHE)2 2,358,198 


post-completion success 
metric Standard V.2.b: Institutions should articulate how they prepare students in a manner consistent with their mission for successful careers, 
meaningful lives, and, where appropriate, further education. They should collect and provide data on the extent to which they are meeting 


these goals. 


pe 
fo) 


Standard 8.6: The institution defines measures of student success and levels of achievement appropriate to its mission, modalities and loca- 
tions of instruction, and student body, including any specifically recruited populations. These measures include rates of progression, retention, 
transfer, and graduation; default and loan repayment rates; licensure passage rates; and employment. 


No, but listed as a potential 
post-completion success 
metric 


New England Association of Schools and pec sagen Aa . 
Colleges, Commission on Institutions of DeerSe os) aQtnie tie wttitions nis, fae, 


Higher Education (NEASC)? Ms, Wet Risen vit 
Northwest Commission on Colleges and Degree-granting institutions in AK, ID, 158 
Universities (NWCCU)'* MT, NV, OR, UT, WA, and Canada 


Southern Association of Colleges and 
Schools, Commission on Colleges 
(SACSCOC)"® 


Set by institu- 


750,450 : 
tions 


E 
oO 


873,911 


fo) 
. 
fo) 
= 
> 


Standard 8.1: The institution identifies, evaluates, and publishes goals and outcomes for student achievement appropriate to the institution's 

mission, the nature of the students it serves, and the kinds of programs offered. The institution uses multiple measures to document student 

success. In doing so, it may use a broad range of criteria to include, as appropriate: enrollment data; retention, graduation, or course comple- Set by institu- 
tion; job placement rates; state licensing examinations; student portfolios; or other means of demonstrating achievement of goals. tions 


Degree-granting institutions in AL, FL, 
GA, KY, LA, MS, NC, SC, TN, TX, VA, and 766 
Latin America 


No, but listed as a potential 


4,359,967 f 
success metric 


Institutions also sometimes identify, evaluate, and publish program-level job placement goals and outcomes as part of Standard 7.3. 


pe 
fo) 


Standard I|.A.14: Graduates completing career-technical certificates and degrees demonstrate technical and professional competencies that 
meet employment standards and other applicable standards and preparation for external licensure and certification. 


Standard II.A.1: All instructional programs, regardless of location or means of delivery, including distance education and correspondence 
education, are offered in fields of study consistent with the institution’s mission, are appropriate to higher education, and culminate in student 
attainment of identified student learning outcomes, and achievement of degrees, certificates, employment, or transfer to other higher educa- 
tion programs. 


Regional (generally degree-granting institutions) 


No, but listed 
as a potential 
success metric 
to publicly 
disclose 


Yes (career and technical edu- 
cation degrees and certificate 
programs) 


Western Association of Schools and 
Colleges, Accrediting Commission for 
Community and Junior Colleges (ACCJC)'® 


Set by institu- 
tions 


2-year degree-granting institutions in CA, 


HI, and the Pacific 130 1,350,663 


Standard |.B.3: The institution establishes institution-set standards for student achievement, appropriate to its mission, assesses how well it is 
achieving them in pursuit of continuous improvement, and publishes this information. 


As part of the annual reporting process, institutions are also asked to report job placement goals and outcomes for each program completed by 
at least 10 students in the designated year and where “reliable data is available.” 


Western Association of Schools and 
Colleges, Senior College and University 
Commission (WSCUC)” 


4-year degree-granting institutions in 


CA, HI, and the Pacific iB? omits ns 


Standard VII.B.1.b: The school demonstrates successful student achievement by maintaining acceptable rates of student graduation and 


Accrediting Commission of Career Schools 


Arts & Sciences (NACCAS) 


of cosmetology arts and sciences 


employment in the career field for which the school provided education...The school supports student achievement rates through student 
transcripts, the school’s verifiable records[,] and documentation of initial employment of its graduates. 


delivery, including a 60% placement rate of graduates. 


70% Cheight- 
ened monitoring 


a Trade and technical programs 370 208,855 Yes (all programs) Yes or reporting or 

iS and Colleges (ACCSC)'® : oe : : 

6 Standard IV.C.3: A school discloses, minimally, the graduation and graduate employment rate for each program offered as last reported to the programmatic 

bo Commission. The disclosure for each program's graduation and graduate employment rate must be accurate, not intended to mislead, and action)” 

ro includes the program population base and time frame upon which each rate is based. 

c 

2 Standard IX.D: The institution establishes and implements written policies and procedures that provide effective means to regularly assess, 70% (heightened 

g Accrediting Council for Continuing Educa- | Non-collegiate continuing education and 62 22.876 Vestuseational prosmanisy Nis document, and validate the quality of the education and training services provided relative to completion and placement rates, as applicable. reporting); 56% 

S tion and Training (ACCET)?° training programs : prog Institutions offering vocational programs provide job placement assistance to graduates and document the results to enhance the effectiveness | (programmatic 

rr of the training services provided. probation) 

= Standard 3-1-110: For the Campus Effectiveness Plan, the following elements, at a minimum, shall be evaluated and reported for achievement of 

3 outcomes, at both the campus and program levels: retention rates; placement rates; graduation rates; the level of student satisfaction; the level | 70% (heightened 

= : es : ue : , 6 

& Accrediting Council for Independent Col- | Institutions that offer programs in profes- 169 172102 Yes (all programs and vee Ol iad lane Sotistactian/ Siclevel te mplover sanstectianednelsHidentl sare ouncomics poate 7 

. 7 : ; : ? ; ani i 

Fy legeoons| school ncle>) sional <eefmeal- aid ocelipartonal elds inStneny) Standard 3-1-704: Each campus shall routinely provide reliable information to the public on its performance, including student achievement warning), 50% 

Ss information, that includes, at a minimum, retention, placement, and licensure examination pass rates (where applicable). The information (adverse action) 

c= provided shall be for the entire campus and for each program as reported to ACICS in its most recent Campus Accountability Report. 

g 

g Standard 3.5: The institution submits accurate and verifiable program placement data each year to the Commission for comparison with 

o ; 9 2 

2 Council on Occupational Education (COE)*” | Career and technical education programs 393 130,751 Yes (all programs) No reguined benchmarks: - oa 

oO 

5 Standard 10.18: The institution provides placement services for all program completers. 

= 

2 60% (height- 
National Accrediting Commission of Career | Postsecondary schools and departments 777 102,289 Vas Gall institutions) Standard 1.D.5: The institution is responsible for the achievement of expected and acceptable outcomes, regardless of mode of educational ened monitoring 


or loss of accred- 


itation) 


*The number of institutions and students are from the National Advisory Committee on Institutional Quality and Integrity (NACIQI)'s May 2018 accreditor dashboard (https:/sites.ed.gov/naciqi/files/2018/05/NACIQI-May-2018-Accreditor-Dashboards.pdf). These counts only include institutions participating in Title IV federal financial aid programs with at least one branch currently operating according to the March 2018 release of College Scorecard 
Beyond footnoted sources, this table was also informed by conversations with accrediting agency staff 


Class: Are cohorts based off of 
starting classes or completing 
classes? If starting, how long after 
a cohort's start is job placement 
measured? 


Exclusions: Are certain subsets 
of students excluded? E.g. those 
unable to seek employment due 
to continuing education, military 
duty, visa restrictions, incarcera- 
tion, medical conditions, or death. 
What about those who are not 
seeking employment? 


Breadth: Must graduates be em- 
ployed in an occupation directly 
in the field for which they were 
trained, in a related occupation, 
or in any comparably or better 
paying occupation? 


Intensity: Is all work counted or 
only full-time, full-year work? 
What about temporary work? 


Timing: How long after program 
completion must a graduate gain 
employment? Must they be em- 
ployed for a certain duration? 


Verification: How must data be 
verified? 


Sample Size: What share of 
graduating students need to be 
included in calculations? 


TABLE 2: JOB PLACEMENT METRIC DEFINITIONS ACROSS NATIONAL ACCREDITING AGENCIES 


Students who graduated from the program within 
"150% of the stated program length" 


"Death, incarceration, active military service deploy- 
ment, or the onset of a medical condition that prevents 
continued enrollment" as well as those "that continue 
on with education in an accredited institution of higher 
education (postsecondary) on at least a half-time basis" 


"The employment is directly related to the program 
from which the individual graduated [and] aligns with a 
majority of the educational and training objectives of the 
program"; may include self-employment; may include 
continuing employment where the program served as a 
catalyst for maintaining or advancing the position 


"Employment is for a reasonable period of time...and can 
be considered sustainable" 


(1) Written documentation from the employer or the 
graduate OR (2) signature of school staff attesting to 
verbal verification with the employer or the graduate" in 
non-self-employment and non-continuing-employment 
cases where the "school can show diligent efforts have 
been made to secure such written documentation with- 
out success"; require third party verification of a random 
sample of data 


Beyond footnoted sources, this table was also informed by conversations with accrediting agency staff. 
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Students who graduated from the program within 
"100% of the published length of the program" 


Death, incarceration, active military duty, serious medi- 
cal illness, or relocation out of the area as well as those 
who waived placement assistance 


"Supporting documentation, including the job descrip- 
ion of a program graduate, must demonstrate that 

he placement is training-related and consistent with 

he vocational objectives, content, and length of the 
graduate’s program”: if continuing or self-employment, 
graduates must attest that they are satisfied and making 
raining-relate income 


Paid; if part-time or temporary, graduates must attest 
hat they are satisfied and making training-relate income 


Employed for 30 days within 120 days after graduation 


Must document "method of verification (e.g. employ- 
er signature, telephone verification with employer, 
telephone verification with student, email with student 
or employer)" 


All; cohort exclusions of more than 15% “will result 
in adjusted waiver and placement rates and will be 
reviewed by the Commission" 


Students who completed the program during the report- 
ing year 


“Pregnancy, death or other health-related issues, 
continuing education, military service, visa restrictions, 
enrollment in English as a Second Language program, 
and incarceration” 


A position in the list of SOC job titles published by the 
institution for which ED approved the program; primarily 
requiring skills listed in the institution’s published 
program description as documented in the employer's 
job description; or in continuing employment where the 
program served as a catalyst for maintaining or advanc- 
ing the position 


Employment must be "intended to be continuing and 
sustainable" 


Must be verified through the Placement Verification 
Program which may require documentation 


Students who completed the program during the award 
year, including those who graduated as well as those 
who left before graduation for related employment 


(1) Those "unavailable for unemployment because of 
situations such as pregnancy, other serious health-re- 
lated issues (physical/mental/behavioral), caring 

for ill family members, incarceration, death, etc.; (2) 
graduate completers who, instead of securing traditional 
employment, are volunteering with a foreign aid service 
of the Federal government, such as the Peace Corps, 

or who are participating on an official church mission"; 
(3) those who refused employment i.e. "failed to keep 
interview appointments, enrolled in the program of 
instruction strictly for personal use, or simply refused an 
employment offer in the field of instruction"; OR (4) are 
waiting to take licensure exams or for exam results 


Those (1) "employed in the field of education pursued or 
in a related field" which may include self-employment, 
(2) continuing education, OR (3) enlisted in the military 


Paid, full time, permanent 


Within a 12-month reporting period selected by the 
institution 


Must be verified by employer or, if self-employed, via 
tax documentation 


Element Needing Clarification ACCSC? ACCET’® ACICS?” NACCAS”? 


“Students scheduled to graduate from the institution 
during the reporting year who graduated, regardless of 
the exact graduation date” 


Those deceased, with a permanent or indefinite disabili- 
ty, deployed for military service/duty, who studied under 
a student visa and are ineligible for employment in the 
United States, and who are continuing their education at 
an institution under the same ownership 


"Employed in a field for which their training prepared 
them (i.e., in a position within the beauty and wellness 
industry that directly relates to their field of training)" 


Paid, permanent 


By November 30 of the year following the reporting 
period 


Must document such as via surveys, telephone logs, 
signed self-certification, emails including documenta- 
tion that the email belongs to the employer/employee, 
or social media screenshots 
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As an illustration of the impact of differences in how accreditors require schools to calculate job placement 
rates, Figure 2 compares how the reported job placement rate of the exact same set of hypothetical stu- 
dents would vary between schools governed by two different national accreditors: the Accrediting Commis- 
sion of Career Schools and Colleges (ACCSC) and the Council on Occupational Education (COE). The result 
is an astonishing 27 percent difference in job placement rates. While the school would report a placement 
rate of 40 percent for the hypothetical cohort of students to ACCSC, the school would report a placement 
rate of 67 percent to COE for the exact same set of students. 


FIGURE 2 
STUDENT OUTCOME (% OF STUDENTS) 
PERMANENT, FULL-TIME EMPLOYMENT: 
GRADUATED WITHIN 150% OF PROGRAM LENGTH (25) 
ACCSC;: 40% COE: 67% 
PLACEMENT RATE PLACEMENT RATE 


PERMANENT, FULL-TIME EMPLOYMENT; 
GRADUATED AFTER 150% OF PROGRAM LENGTH (10) aN 


PLACED (30) PLACED (60) 
PART-TIME AND TEMPORARY EMPLOYMENT (5) 
NOT PLACED (45) ; NOT PLACED (30) 


CONTINUING EDUCATION/MILITARY (25) 


EXCLUDED FROM EXCLUDED FROM 


COUNT (25) COUNT (10) 
VOLUNTEER WITH A FOREIGN AID SERVICE OF THE 
FEDERAL GOVERNMENT (10) 


OTHERWISE UNEMPLOYED (25) 


Finally, some career programs within institutions receive separate program-level accreditation through one 
of many programmatic, or specialized, accreditors that may have their own employment requirements.*° 
These programmatic accreditors range from the Commission on Massage Therapy Accreditation to the 
American Bar Association. While programmatic accreditors often set higher standards for particular pro- 
grams, this system creates further confusion for students as it is possible for a program to be part of a 
school with national accreditation but lack the relevant programmatic accreditation, leaving students who 
complete the program unable to land a job in their field of study. 
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n part because the U.S. Department of Education (ED) is explicitly prohibited from specifying, defining, or 
prescribing any specific standards, the definitions and requirements for reporting job placement rates are 

eft entirely up to accrediting agencies.*' One result is the tremendous amount of inconsistency between 
accreditors regarding the calculation and reporting requirements. A second and equally problematic issue is 
that job placement rates are self-reported by institutions and have proven subject to manipulation and gam- 
ing by unscrupulous schools. National accreditors have been accused of failing to adequately audit reporting 
of job placement rates and multiple institutions have also been forced to pay millions of dollars to settle 
claims that they inflated or lied about job placement rates, as further discussed below. 


State Governments 


In addition to accreditors, states governments also approve and regulate higher education institutions that 
operate within their jurisdictions.** States are increasingly using employment metrics in postsecondary 
funding and oversight decision-making. However, like accreditors, states use widely varying definitions. 


Many states have developed, or are developing, state longitudinal data systems that match reported post- 
secondary education data with state workforce data to calculate employment outcomes, discussed in the 
next section. A few states also require that certain programs meet specific job placement or completion 

benchmarks to receive approval to operate within their state.°? 


A few states actually tie a portion of state funding to employment metrics. Historically, states have allocat- 
ed funds largely based on historical patterns or inputs such as enrollment, often referred to as incremental 
budgeting. In recent decades, however, an increasing number of states have begun including student out- 
comes in their calculations, referred to as outcomes-based funding (OBF). As of fiscal year 2018, half of all 
states award at least some amount of higher education funding via OBF, including the seven states detailed 
in Table 3 that specifically use some type of labor market metric for this purpose.** Other states, such as 
Texas, use related earnings metrics.*° However, as with accrediting agencies, such measures differ across 
states. For instance, Florida uses a threshold earnings rate that only counts jobs that pay at least $25,000, 
while Louisiana only counts those in certain occupations. 
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TABLE 3: EMPLOYMENT METRIC REQUIREMENTS ACROSS STATE 
OUTCOMES-BASED FUNDING MODELS 


Employment Rate Metric 
Publicly 
of OBF 


and improvement) 


Percent of bachelor's graduates con- 1 of 10 equally 
State University | tinuing their education or employed and weighted metrics 
System (4-Year)*° }earning $25,000 or more one year out |State unem- tied to over $500 
(level and improvement) ployment million of OBF 
insurance (UI) 
Percent of graduates employed or data , ely ; 
College System?’ |continuing education one year out (level NGIEINeG meliics 
tied to $60 million 
Florida 
Number of completers in programs orl) nies (ed 
. i P . ia 8 Not specified to 17.5% of higher 
leading to “4 or 5 star jobs f : 
liguisiana education funding 
5% of higher ed- 
Whether or not there was a 5 percent ucation funding is 
State Colleges : : ii rs ; 
, ce increase in the “related-employment Not specified held back until the 
and Universities i 
rate school meets 3 of 
Minnesota®? 5 goals 


a 38 _ 
Total degree and certificate completers 
competitively employed, serving in the 
2-Year roiliteny, erenaiies ae ey iouleyear Not specified | 1 of 6 equally 
nstitution, or found in state wage : : 
: ; weighted metrics 
records following graduation (level and : 
: generally tied to 
mprovement) h ‘arity of 
Job placement 180 days out (level and Me male ty ove 
Technical , Not specified higher education 
mprovement) : 
funding; although 
Students employed full time, participat- | National metrics were tabu- 
ing in a volunteer or service program, Association of |lated, no OBF fund- 
Adis serving in the military, or enrolled in Colleges and ing was allocated in 
continuing education in the six months |Employers First |FY18 and FY19 
following graduation (level and im- Destination 
Missouri provement) Survey 
x 
42 
x 
in43 


: : ; Number of students who are employed 
bee bie Daverieyrot following degree or certificate comple- Not specified 
NY (2-Year) : 
New York“! tion 
The number of placeable graduates 5-15% of OBF, 
from the spring, summer, and fall terms which represents 
Public 2-Year within a calendar year who obtainem- |Not specified |about 80% of 
ployment in a related field through June higher education 
Tennessee 30 of the following funding 


The number of graduates who report 
: they ale WOT ARENDS related ia Graduate fol- —_}1 of 10 metrics tied 
Technical College |their program of study divided by the fae 
low-up survey |to over $26 million 
System total number of respondents who are ee of OBE 
employed (both in related and non-re- 
Wisconsin ated jobs) 


Institutions may also need to report employment metrics to states under the Carl D. Perkins Career and 
Technical Education Improvement Act, which provides $1.1 billion in federal funding to career and technical 
programs and high schools and colleges.** Reauthorized in 2018, the Perkins Act gives state agencies the 
power to allocate funds to institutions according to state-defined core indicators, including the percentage 
of career and technical education completers who are, among other outcomes, “placed or retained in em- 
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ployment.” Although some parameters are specified by the federal government—for instance, employment 
must be assessed “during the second quarter after program completion” —others are left up to states to 
determine.*° 


While states are increasingly looking to employment metrics, particularly job placement rates, in making 
decisions about where to allocate both state and federal higher education resources, variation in the criteria 
that individual states use has led to data that cannot be compared across states. 


The Federal Government 


Finally, the federal government requires its own array of employment metrics across multiple federal efforts, 
including the College Scorecard, the gainful employment regulation, the Workforce Innovation and Opportu- 
nity Act (WIOA), and Title IV requirements for short-term programs. 


£ 


The most comprehensive, easily available employment information at the federal level was the threshold 
earnings rate formerly available on ED's public-facing College Scorecard tool. Together with the median 
earning rate, this metric provided new insights to consumers. The institution-level threshold earning rate 
measured the share of former students who received federal financial aid who earned at least $28,000 six 
years after entering college. The income threshold was based on earnings among 25- to 34-year olds who 
self-reported high school as their highest level of education. Stating concerns about the utility and validity 
of the income threshold, ED removed the threshold earnings rate from the public facing tool in September 
2018.*° This eliminated a key source of easily accessible information for students and others. 


D does still make the threshold earnings rate data available for download, including eight and ten years 
ter students entered college. It also makes available for download—but never published on the consum- 
r-facing Scorecard tool—an overall employment rate that measures the share of student aid recipients 
mployed and not employed six, eight, and ten years after entering college. Figure 2 shows the six-year 
employment rates by school level for the most recent cohort. Overall employment rates—calculated as 
the share of former students with any positive earnings in a given year—range between 80 and 90 percent 
across sectors. However, only 34 to 61 percent of students are earning more than $28,000 six years after 
entry. 


Oo ov m 


FIGURE 3: SHARE OF FORMER TITLE IV STUDENTS EMPLOYED 6 YEARS AFTER ENTRY 


E] Earning Less Than or Equal to $28,000 


El] Earning More Than $28,000 


Employment Rate 


Public Nonprofit For-profit 


Data are for the pooled cohort who enrolled during the 2007-08 and 2008-09 academic years 
DATASOURCE: College Scorecard, Department of Education (2018) 
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Current Scorecard data are available at the institution-level due to data availability at the time it was creat- 
ed; however, both the previous and current Administration stated intentions to disaggregate data by pro- 
gram.*” Program-level data are critical to decision-making, as there is substantial program-level variation in 
employment outcomes.*® 


Similarly, ED originally used entry cohorts due to concerns about the accuracy of graduation data report- 
ed by institutions, but it has since worked to resolve these concerns.*? Many students do not choose their 
programs until partway through their studies, making program data collected at entry less meaningful. 
Moreover, as students from the same entry cohort often graduate at different times, comparisons across 
exit cohorts are easier to understand. Finally, using exit cohorts allows for more immediate comparisons. ED 
began collecting information on the program students are enrolled in during the 2014-15 award year, mean- 
ing it can already compute employment outcomes for students one year after program completion, but it 


will not be able to compute employment outcomes for students six years after program entry until 2021.°° 


A second federal policy that utilizes employment metrics is the Gainful Employment regulation (GE reg- 
ulation). Under the HEA, higher education programs must either lead to a degree at a public or nonprofit 
institution or “prepare students for gainful employment in a recognized occupation” in order to receive Title 
V federal student aid funds.*' In 2014, as part of its regulatory effort to define what was required to demon- 
strate that a school’s graduates were prepared for gainful employment, ED required schools to “report job 

placement rates at the program level if the institution is required by its accrediting agency or state to calcu- 


ate a placement rate” as well as disclose these rates “to enrolled and prospective students.”°? 


The GE regulation applies to non-degree programs at any institution and all programs at for-profit institu- 


tions except certain bachelor’s degrees in liberal arts.°?Under the regulation all non-degree programs at any 
institution and all programs at for-profit institutions that are nationally accredited (except certain bachelor’s 
degrees in liberal arts), must both disclose job placement rates and must provide those rates to prospec- 


tive students. Notably, ED has proposed eliminating these GE disclosure requirements as early as January 
2019.%4 


Because GE rules only require institutions to report job placement rates that they are already required to 
report to states and accrediting agencies, even when fully implemented, definitional and calculation incon- 
sistencies at the state and accreditor level persist. For example, the website for a vocational nursing certif- 
icate at Brightwood College's location in San Antonio, Texas reports that according to the state's required 
calculation, 62 percent of program graduates were placed, whereas according to the accreditor’s (ACICS) 
required calculation only 33 percent of graduates were placed.°° This discrepancy is particularly troubling 
given the fact ACICS lost its status as an accreditor over concerns that it failed to ensure that institutions 
under its control were in fact preparing students for gainful employment and accurately reporting job place- 
ment rates. 


Another federal employment metric is set out in the Workforce Innovation and Opportunity Act of 2014 
(WIOA). WIOA requires states to strategically align six federally-funded workforce development programs: 
employment and training programs for adults; employment and training programs for dislocated workers; 
employment and training programs for youth; employment services; adult education; and vocational reha- 
bilitation. These programs are required to report, among other “common measures,” the share of former 
tudents employed two and four quarters after program exit.°° 


A final federal employment metric applies to short-term programs between 300 and 600 clock hours. For 
students attending these programs to receive Title |V aid, 70 percent of program completers must have held 
a job in their field, or a related one, for at least 13 weeks within 180 days of program completion.?’ Due to 
limitations in existing federal data, the number or share of short-term programs that elect to pursue federal 
loan eligibility and are thus affected by this requirement is unknown. 
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Overall, federal employment metrics, similar to state and accreditor metrics, fail to provide comparability. 
While the College Scorecard’s threshold earnings provided important new information that was comparable 
nationwide, the recent elimination of this information represents a step backwards. 


CURRENT EMPLOYMENT METRICS ARE NOT ACCURATE, COMPREHENSIVE, AND VERIFIABLE 


Beyond comparability challenges across accrediting, state, and federal agency requirements, the accurate 


collection and verification of employmen 
ous concern. In addition to being consist 


4 


help students but can cause substantial 


As summarized in Table 4, employment 
methods—institutional surveys, a sta 
strengths and weaknesses when it co 


4 


metrics can be collected and veri 
te data match, and a federal data match—each of which has its own 
mes to the accuracy and verifiability. 


4 


TABLE 4 : DATA COLLECTION OPTIONS 


Query students for documenta- 
tion potentially including survey 
responses, business directory 
listings, websites, business 
cards, written statements from 
the student's employer, state 
and federal income tax forms, 
evidence of Social Security pay- 
ments, etc. 


State longitudinal systems that 
link postsecondary and UI sys- 
tem workforce data 


Federal system that links post- 
secondary and federal workforce 
data (e.g. LEHD, NDNH, SSA, or 
IRS data) 


- Flexible data collection capabil- 
ities including variables collected 
and frequency of data collection 


- Can collect data on all students, 
regardless of whether they move 
states after attending school 


- Data is collected quarterly 


- Lower administrative and finan- 
cial costs for institutions 


- Lower administrative and finan- 
cial costs for institutions 


- Can collect data on all students, 
regardless of whether they move 
states after attending school 


- Potentially decades of historical 
data 
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t metrics—and job placement rates in particular—remains a seri- 
ently defined across programs and schools, employment metrics 
must be verifiable and accurate to be useful. When employment metrics are inaccurate, they not only fail to 
harm by misinforming students drowning out other, more accurate 
metrics; and ultimately influencing students to invest in and devote time to a program on the premise that 
their odds of employment are much higher than they actually are. 


fied using one of three main 


- Weak enforcement and verifi- 
cation, creating high potential for 
incorrect data, misrepresentation, 
and fraud 


- Low student response rate 


- Greater administrative and 
financial costs for institutions 


-Do not always include students 
who move out of state, work out 
of state, or are ineligible for state 
UI 


- Most do not include all schools 
in the state 


- Most do not track occupations 


- Only keep only a few years of 
historical data 


- Currently limited only to 
students who borrow to attend 
college 


- Does not comprehensively 
record employment intensity, 
occupation, and many variables 
needed to calculate cohort exclu- 
sions 


- Data is only collected annually, 
making it difficult to determine 
employment timing 


Strengths and Weaknesses of Institutional Collection of Employment Metrics 


The first way to calculate employment metrics is for institutions to directly survey their alumni, and possibly 
verifying information via business directory listings, websites, business cards, or some other method. Many 
institutions use such methods to comply with national accrediting agency requirements. However, surveys 
generate large administrative and financial costs to institutions. One 2004 study on WIOA evaluations 
found follow-up surveys cost approximately $13.25 per participant compared to only $0.05 per participant 
via automated data-matching.°° 


Moreover, surveys have a large potential for deliberately or intentionally inaccurate or misleading data. Un- 
scientific alumni surveys may exhibit reporting bias, underscoring the importance of getting good response 
rates or testing for nonresponse bias. For instance, employed individuals may be more likely to respond.*? 


Some institutions that are required to report employment metrics and meet certain thresholds may also 
mischaracterize employment rates given limited resources or perverse incentives. While some private 
sector companies offer third party placement rate verification for institutions, it is unclear how expensive or 
effective these services are. 


In recent years, job placement rates have become notorious for being inaccurate or misleading. In 2012 an 
ED working group concluded that “state and federal investigations have found that several institutions have 
provided false job placement rates to consumers, presumably to persuade consumers to enroll in the pro- 
gram."°! And in the 2014 GE regulation, ED noted that “several state Attorneys General have sued for-profit 
institutions to stop...fraudulent marketing practices, including manipulation of job placement rates."° 


For example, in 2013, Career Education Corporation (CEC), a large for-profit education company that 
operated multi 


tiple different online and campus-based colleges and enrolled over 100,000 students, agreed 
to pay $10.25 million in a settlement with the New York State Attorney General following allegations that 
it routinely and systematically inflated job placement rates. The school was accused of counting work at 
one-time, one-day health fairs, some of which were held at the request of the school, as employment and 
counting retail sales jobs as in-field placements for graduates of criminal justice programs.® At the CEC- 
owned California Culinary Academy in San Francisco, “former students alleged that the college’s admissions 
representatives and catalog boasted a job placement rate of 97 percent, but that the college did not tell 


applicants that the statistics included graduates working as baristas, prep cooks, line cooks and waiters, jobs 
for which no degree was necessary.”°4 


In a separate matter, ED found that Corinthian Colleges, Inc.'s Heald College engaged in widespread mis- 
leading and deceptive conduct in reporting placement rates, including paying temporary agencies to hire 

ts graduates to work at short-term, temporary jobs at its own campuses and counting these graduates as 
placed. The company also counted placements that were clearly not within the student's field and counted 


as placed graduates whose employment began prior to the graduate’s attendance at Heald.°° 


Similarly, DeVry, Inc. claimed that 90 percent of graduates actively seeking employment found jobs in their 
field within six months of graduation and that their graduates’ incomes one year after graduation were 15 
percent higher than those of graduates from other colleges or universities. An investigation by the Federal 
Trade Commission (FTC) found these claims were misleading and obtained a settlement of $100 mil- 

lion. The FTC also alleged that DeVry made false claims about graduates it counted as placed in the field 
of study. These included “a graduate from the technical management degree program working as a mail 
carrier”, “a business administration graduate working as a waiter at the Cheesecake Factory”, “a business 
administration graduate working as a secretary at a prison”, and “a technical management graduate working 
as a Sales associate at Macy’s.”°” 
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Table 5 documents investigations of a dozen for-profit colleges nationwide that resulted in millions of dollars 
paid by colleges to settle cases involving allegations that they deceived and mislead prospective students 
about job placement rates. 


TABLE 5: EXAMPLES OF SETTLEMENTS RELATED TO MISLEADING OR 
FALSIFIED JOB PLACEMENT RATES 


Settlement y 
Year| School Allegations Related to Job Placement 
Amount 

2009 Alta Colleges (Westwood $7 million Misrepresented compliance on job placement reporting requirements 

College) to the state 

Alta Colleges (Westwood Misrepresented job placement to the public by counting freelance jobs 
2012 5g $4.5 million 

College) that only lasted a few days as employment 


Career Education Corpo- Misrepresented job placement to the public by (i) counting work at 

ration (Le Cordon Bleu, single one-day health fairs, some of which were held at the request of 

Brown Mackie, Colorado the school, as employment and (ii) counting retail sales jobs as in-field 
$10.25 million be Dn st acca 

Technical University, placements for graduates of criminal justice programs, among other 

American Intercontinental improper methods. Disclosed rates of 55-80% when actual rates were 


University)”! 24-64%. 


2013 oes : ar : arate 
Art ehterpnses? $3.7 million Fraudulently maintained Title IV eligibility by misrepresenting job 
placement rates. 


Misrepresented job placement to the public by counting jobs in fast 
Sullivan and Cogliano $425,000 food and big box stores as in-field placements for a program that trains 
Training Centers”? for work in medical offices. Disclosed rates of 70-100%, when actual 
rates were less than 25%. 


Misrepresented job placement to the public: among other improper 
Education Management methods, they “(i) falsely represented the existence of a job with an 
Corporation, Inc. employer despite evidence the student was not so employed; (ii) fal- 
(Art Institutes, South $4.4 million sified job titles; (iii) counted jobs that were not in the students’ field of 
University, Argosy Univer- study; (iv) counted short-term jobs as permanent...(vi) used incorrect 
sity, and Brown-Mackie verification forms and (vii) counted jobs graduates held before graduat- 
College)” ing from, or even enrolling in, NEIA”. Disclosed rates of 89-98%, when 


actual rates were 26-60%. 


erence Education Gril Misrepresented job placement to the public by characterizing tempo- 
45 P $3.75 million rary jobs, part-time jobs, and jobs outside the field of study as full-time 
moglter College) in-field employment. 


Lincoln Education Services Misrepresented job placement to the public by characterizing tempo- 
Corporation (Lincoln Tech- $1 million rary jobs, part-time jobs, and general retail jobs as in-field placement 
nical Institute)’ for graduates of a criminal justice program. 
Misrepresented job placement to the public and accrediting agencies 
Corinthian Colleges, Inc. $11+ billion and failed to comply with applicable state and federal regulations and 
(Everest and Heald)’® accreditor standards. Disclosed rates were up to 85 percentage points 
higher than actual rates. 
DeVry Education Group Misrepresented job placement to the public, such as by characteriz- 
(Now Adtalem Global $100 million ing jobs as in-field when they were not and excluding some students 
Education, Inc.)’? actively seeking employment from their cohort. 


Misrepresented job placement to the public by characterizing appren- 
Flatiron School®° $375,000 tices, contract workers, and self-employed freelance workers, some of 
which were employed for less than 12 weeks, as employed. 
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7 Education Affiliates”® $13 million Misrepresented job placement to the public. 


Strengths and Weaknesses of State Level Reporting of Employment Metrics 


A second method to accurately collect and verify employment data is through a state data match in states 
with sufficiently robust data systems. State and federal data matches rely on administrative data, which is 


collected by governmental agencies to administer s 


pecific programs. Under a state data match, state post- 


secondary data is linked with state workforce data, often through a state longitudinal data system (SLDS). 
These workforce data are generally from state unemployment insurance (UI) records, which are collected 


quarterly from employers and include wage and ind 
data. 


ustry information that is not available through federal 


+ 


The Educational Technical Assistance Act of 2002 was created to award states grants of up to $20 million 


to build SLDSs, and has been called the “single bigg 


est driver” in state data system development over the 


past 10 years.®’ A 2018 survey conducted by State Higher Education Executive Officers (SHEEO) found that 
35 states specifically link postsecondary and workforce data while another seven are planning a linkage.®? 
Similarly, a 2016 survey of state agencies by the Workforce Data Quality Campaign found that 28 states 
report specifically matching state UI data with student data to calculate employment metrics.2? Moreover, 


4+ 


most states take the additional step of disaggregating matched data to calculate the wages of graduates 


£ 


program, which is currently funded at about $32 mi 


State data matches face their own weaknesses. Ma 


4 


ittle as three years.® 


or in the military.8° These limitations are significant 


ted out of state in 2011.8° Overall, up to 18 perce 


+ 


4 


mployment metrics.%° 


4 


Classification System or NAICS codes) and not occ 


State Ul systems also do not continue to track graduates who move out of state; work in a state di 
from where they live; or are ineligible for state Ul, such as those who are self-employed, federal employees, 


@ 
ikely to move between states. Thus, only counting former students who stayed in state may underestimate 
e 


tatus by quarter, or employment intensity, which may be necessary for computing certain definitions of 
employment. For instance, state Ul agencies tend to collect industry data (e.g., North American Indus 


of each program. However, the Trump Administration's Fiscal Year 2019 budget proposed eliminating the 


llion annually. 


ny states do not currently include data on private 


4 


schools. Indeed, only nine states currently have SLDSs with program-level employment outcomes for all 
institutions, as detailed in Table 6.* Additionally, state Ul systems generally maintain fewer years of histori- 
cal data than federal workforce datasets. Although some keep data for over 10 years, others keep data for as 


£ 


ferent 


given that over 2.5 million employed Americans moved 


rom one state to another between 2016 and 2017 alone,®’” and another estimated 5 million Americans com- 


nt of the civilian labor force alone was not employed in 


Ul-eligible jobs as of 2010.8? Discounting such individuals is particularly problematic as research finds that 


ducational outcomes and interstate mobility are correlated, and those with more education may be more 


Many SLDSs do not include useful variables such as occupation, credential completion date, employment 


+ 


try 
upation data (e.g., Standard Occupational Classification 


or SOC codes). Although industry and occupation often align, many occupations are spread across indus- 
tries, such as those in computer science. Similarly, only eight states collect hours or weeks worked through 
their Ul systems, and only five include hours worked in their longitudinal data systems. 


One of the most critical ways in which state systems could help to improve the accuracy and veri 


job placement rates is through the increased use of 


lation requires that all covered programs must provi 


fiability of 
SOCs as part a state's Ul wage records. The GE regu- 
de ED a list of “occupations that the program prepares 


students to enter if the student completes this program of study,” as well as the SOC codes associated 
with each of these occupations.”' Thus, state SOC data could be used to compute the number of program 


graduates employed and earning wages in occupati 


+ 


ons for which they were trained and thus creating a 


new means of verifying job placement rates. Curren 


tly only Nevada, North Dakota, Oklahoma, and Virgin- 


ia include occupational codes in their SLDSs. However, these states generally collect these data through 
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burdensome means such as surveys. Alaska and Louisiana are currently collecting occupation codes as part 
of their Ul wage records. While neither includes these in their SLDSs, the DOL’s Bureau of Labor Statistics is 
also currently conducting a data-sharing pilot program that will analyze the utility of Alaska and Louisiana’s 


expanded records.” 


Federal and state pilots may enable the measured expansion of state UI systems. While experts recently 
concluded that states and employers are not currently prepared to add variables to Ul wage records nation- 
wide, in fact approximately half of all states report they are adding, or have the capacity to add, variables to 
their Ul records. Increased federal and state investments would facilitate and speed such development. 
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TABLE 6: DATA ELEMENTS COLLECTED BY STATE LONGITUDINAL DATA SYSTEMS” 


Postsecondary Data Labor 


California 


X 


— 
es 
a 


Division of Colleges 


Maine 
Maryland 
Massachusetts 


Michigan 
Minnesota 


Mississippi 
Missouri 
Montana 
Nebraska 


North Carolina 
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TABLE 6: DATA ELEMENTS COLLECTED BY STATE LONGITUDINAL DATA SYSTEMS” 


Types of Institutions Covered Postsecondary Data Labor 


é Quarter 
Program/Major Eabeod Year Employed SOC Code 
ee ee ee 


Private, 
For-Profit 


Private, 
Nonprofit 


Degree or 


4-Year Public Certificate Awarded 


2-Year Public Date Awarded 


Ohio 


SCHEV 
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Washington SBTC* 
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Wisconsin 
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# e UWYO* 
Number of 
States 
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N 
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As described in Table 7, several initia 


may use the system, these data may 
by law, data are limited to just a few 


+ 


+ $ 


four states to ten states, but 


itate wage record data-sharing between states. H 


MLDE) supported by the Bill & Melinda Gates Found 


. 


tives have attempted to bridge such gaps in state databases, but holes 
remain. The DOL operates the Wage Record Interchange System (WRIS) and similar VWWRIS2, which facil- 


£ 


only be used 


owever, only state agencies operating WIOA programs 
or performance reporting if this reporting is mandated 


variables and just the past two years of information, and seven states 


+ 


+ 


do not participate in WRIS2.°4 Some states have also joined regional data-sharing agreements, such as 


the Western Interstate Commission for Higher Education Multistate Longitudinal Data Exchange (WICHE 
ation. WICHE MLDE is currently being expanded from 
it remains far from nationwide.” 


Beyond interstate systems, 43 states and the District of Columbia participate in DOL's Federal Employment 


Data Exchange System, which connects state agen 


cies with Department of Defense (DoD) and Office of 


Personnel Management (OPM) data on federal and military employees, enabling the inclusion of these co- 


the feasibility” of the system.?” 


ment of Commerce Census Bureau's 


a promising tool for improving labor 
LEHD to publicly provide comprehen 


also soon be releasing employment outcomes 


Colorado.'°° Similarly, states provide 
Directory of New Hires (NDNH), wh 
in locating parents and enforcing chi 


limit data usage and sharing agreements m 


Finally, most states submit state Ul data to two federa 


combines state U! data with federal census, su 


datase 


horts of employees in state Ul datasets.°° However, this system is currently suspended, as DOL “reassesses 


4 


ts. First, states submit data to the U.S. Depart- 


Longitudinal Employer-Household Dynamics (LEHD) systems.?8 LEHD 


£ 


d support orders.'°! 


There are many ways the federal and state 
secure data-sharing across state Ul system 
opt into regional data-sharing agreements 


£ 


Ho 


ust be individua 


rvey and administrative data to analyze employment and is 


+ 


market data. The University of Texas System recently partnered with 
sive program-level employment outcomes on its graduates.2? LEHD will 
or certain institutions, such as public colleges in Texas and 
data to the U.S. Department of Health and Human Services’ National 
ich combines various datasets to assist state child support agencies 


wever, currently both LEHD and NDNH strictly 
ly negotiated.'°? 


governments can close data gaps among systems that facilitate 


s and between t 
or WRIS data. The federal government can use WIOA regu- 


hese systems and federal datasets. States can 


lations to allow education agencies to use WRIS and WRIS2. It can also build processes whereby states 

and institutions can access linked federal datasets, such as the University of Texas and LEHD partnership. 
Similarly, previously introduced legislation would allow specified federal agencies and their designees to use 
NDNH data for research associated with assessing positive labor market outcomes.'? 
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TABLE 7: SYSTEMS THAT AUGMENT STATE UI SYSTEMS™ 


Additional Scope 


Ul-eligible workers in 39 states, 
DC, and Puerto Rico (WRIS2) and 
nationwide (WRIS) 


Wage Record 
Interchange System 
(WRIS and WRIS2) 


Exchanges data for 
past 2 years 


DOL facilitates data exchange 
between state Ul systems 


Regional Data-Sharing 
Systems e.g. WICHE 
MLDE 


States facilitate data exchanges 


between thee Ulevstems Currently 4 states (WICHE MLDE) Specific cohorts 


Federal Employment DOL facilitates data exchange Federal and military employees, 
Data Exchange System | between state UI systems, DOD, | except US Postal Service workers, 
(FEDES) and OPM in 43 states and DC 


Exchanges data for 
past 2 years 


‘ : OCSE combines state UI data 65 
National Directory of sah dodersldatatie aac crate Federal and military employees 


New Hires (NDNH) ; : nationwide 
child support agencies 


Longitudinal Census combines state UI data Co Over a decade, 
i Civilian federal and self-employed ‘ 
Employer-Household with federal census, survey, and ae eee | continually aug- 
Dynamics (LEHD) administrative data pe mented 


Overall, serious gaps remain in state Ul records and SLDSs. However, with additional investments to address 
the inclusion of new cohorts, addition of SOC codes, and gaps in data sharing between states and between 
states and the federal government, these databases offer promising opportunities to improve the accuracy, 
verifiability, and accessibility of employment data. 


Strengths and Weaknesses of Federal Level Reporting of Employment Metrics 


A third employment data collection option is a federal data match, whereby federal postsecondary data is 
inked to federal workforce data, most likely from LEHD, NDNH, the Social Security Administration (SSA), or 
the U.S. Department of Treasury Internal Revenue Service (IRS). A federal student-level data network could 
further streamline a federal data match by facilitating secure data agreements across agencies. 


The SSA collects data for tax records on employees from businesses through the W-2, while the IRS col- 
ects data on self-employed individuals through Form 1040. SSA and IRS data are shared between the two 
agencies: these agencies’ relative analytic capacity has impacted which agency has participated in past data 
matches with ED.'°° For instance; institutions calculate GE-mandated debt-to-earnings ratios by providing a 
ist of program completers to the SSA to be matched with wage data.'°° Meanwhile, the College Scorecard 
uses federal education and workforce data matched by the Treasury. However, federal SSA and IRS data are 


only collected annually and do not record employment intensity, occupation, and a number of other vari- 
ables needed to calculate cohort exclusions, making it difficult to determine employment timing. 


CURRENT EMPLOYMENT METRICS ARE NOT SUFFICIENTLY ACCESSIBLE OR TRANSPARENT 


Even if they were comparable and accurate, employment metrics cannot influence student decisions unless 
they are transparent and easy to access. To meet their potential, employment metrics must be easily acces- 
sible on a public platform and prominently disclosed by colleges. 
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Research on public disclosures finds tha 
compliance in disclosing required inform 
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and understandable, and avoid overwhe 


are acting or be memorable; a 
information.'°? Consumer t 

tration for nutrition labels—can help ach 
Currently ma 

not require them to publicly d 

job placemen 

One way such requirements c 


such the College Score 


come 


Education & Training P 


score 


bases of job placen 


Score 


Only 


carefully written and enforced, are they ful 


are di 


disclosures required by HEA at 152 public and private four-year sch 
schools were not in compliance with job p 
institutions collected a 
from the college’s hom 


sible. 


Some states attempted to address this problem by creating sta 


s and resulting fu 


aceme 
cards for student 
nent 


card, there is currently n 


when requiremen 


4 


that are 


sclosed in ways 


| discl 
epage 
Moreover, some schoo 


V 


card. States with outcomes-based 
nding, and some have more robust 


ts and workers, and some states and accred 
disclosures."° However, because the inform 


ts for the disclosure of 


osures onaSi 
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esting and standardized guidance—as done by the U.S. Food and Drug Adminis- 


ieve these goals.'°° 


ny states and accrediting agencies that require institutions to report employment metrics do 
isclose these metrics. The GE rule requires that institutions publicly disclose 
t rates for career education programs, ED has proposed eliminating it. 


an be met is by publishing employment metrics on a central public platform, 
unding typically publish institution-level out- 
consumer reporting systems such as the Florida 
nt Information Program.'°? Today, 23 states report publishing their own college 
iting agencies also create centralized data- 
ation has been removed from the federal 
formation in a usable format. 


o single platform that provides this in 


stitutions at the national level are 
red disclosures are often buried or 


this information by in 
seful. Currently, requi 
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easy to overlook, vague, or not fully accurate. For instance, a 2011 survey of 
ools found that at least 17 percent of 
acement disclosure requirements. Although half of surveyed 
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ngle, standard page, these pages “were rarely accessible directly 
alf had no central page, making the information even less acces- 
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tories or the share of students who intend to 
1 


' The other h 


+ 


ewide databases of GE disclosures. For 


instance, the Texas Higher Education Coordinating Board publishes GE data on its Texas Higher Education 


Data portal.'* And some accrediting agencies include links to d 
However, such systems are exceptions and not the norm. 


Overall, the public does not have consistent access to clear and 


isclosures in their directories of institutions." 


accurate employment metrics. Systematic 


changes are needed to empower students and others to make better decisions. 


EMPLOYMENT METRICS CAN PROVIDE KEY INFORMATION 


Although students and their families consider many factors when deciding whether and where to attend 
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year colleges found that 84 percent identified the ability to get a better job 
heir decision to attend college, a larger share than for any other factor cited.'4 
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duates who are able to get a good job."° Beyond students and families, other 
ble to evaluate and compare colleges and programs when deciding where to 
ion resources. 


ngle metric is perfect, an 
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TRICSA 


d employment metrics are no exception. Employment metrics do not capture 
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postsecondary education. Moreover, some students may not enroll in higher education for economic rea- 


sons at all. In such cases, employment metrics are not an accurate way to measure whet 


reach their goals. 


Additionally, like many 


program provides, but rather the overall success of its former students. Employment met 


by a number of variables other than the performance of the college. Similarly, employment metrics do not 
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her these students 


metrics, employment metrics do not capture the added value a higher education 


+ 


rics are influenced 


measure whether or not a program “pays-off" or provides a positive return on investment, like other metrics 
that incorporate the costs of higher education or compare a student's earnings before and after enrollment. 


Additionally, some credent 
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£ 


live within 50 miles o 


sense for programs th 


+ 


may offer substantial i 


+ 


ixed income threshold. 


metrics and help fill exi 


Finally, specific employment met 
at prepare students for certain occupations. Threshold earnings measures have their 

own drawbacks. Because cost of 

rural locations, econom 


+ 


that they might in anot 


Nonetheless, employment metrics can provide key insights that com 


sure the share of students 
such as repayment or coh 


borrow federal student loans. Furthermore, as employment increases the ch 


epayment, employme 


=I 


Overall, employment n 
families, policymakers, 


invest time and resources i 


nt metrics 


netrics tha 
and other 


must be consistently d 


efined, col 


arket value. For exa 
n the workforce than other credentials, but can in 
pleting a bachelor’s degree with substantial economic value."® 


sting infor 


ials that may lack direct labor market value are a 
mple, an associate's degree in liberal arts or trans 


Also, as with any data, information alone is not always sufficient to alter behavior. Choices about 
attend college are sometimes limited by proximity: a recent study found that one third of Americans do not 
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fer studies 
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come increases for students and value to society, bu 
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plement other common labor market 
mation gaps. Unlike earnings metrics—such as mean or median earnings, 
the difference in earnings before or after enrollment, or returns on investment—employment metrics mea- 
meeting a basic threshold of success. Additionally, unlike student loan metrics— 
ort default rates—employment metrics are not necessarily limited to those who 
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t are clear, accurate, and transparent have the potential to help students, 
stakeholders make more informed decisions about where and when to 
n higher education. But in order to provide this benefit, the employment metrics 


trustworthy manner, and easily 


accessible by students and other decision-makers. Students are relying on colleges, accreditors, and state 


and federal policymakers n 


RECOMMENDATIONS 


Current employment metrics can and should be made more consistent, accurate, and accessible. In par- 
ticular, both job placement rates and threshold earnings rates have tremendous potential to help students, 
policymakers, and others understand the share of students succeeding at various programs, yet neither em- 
ployment metric can currently be used for this purpose. In order to maximize the potential of both metrics, 
several policy changes should be undertaken. 


naking this a reality. 
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Job Placement Rates 


First, more accurate and comparable job placement rates are a critical tool when considering programs 
geared toward preparing students for a specific occupation. Such rates can tell students and others how 
often students from a program secure jobs in the specific occupation. But current job placement rate defi- 
nitions vary wildly across accrediting agencies, state, and federal regulators, including which students are 
counted in the placement rates, how long they must be employed, how much they must work and what 
occupations are within the field of study of the program. 


n order to establish a standardized, robust, and consistent definition for job placement rates across the tri- 
ad, Congress should authorize ED to set standards for accreditors. The HEA currently places restrictions on 
the ability of the federal government to set such standards. Only if the federal government is able to take the 
ead in bringing state and accrediting bodies together to establish consistent criteria, is it likely that a stan- 
dardized job placement definition can be established. Some accreditors additionally establish benchmarks 


for job placement rates to evaluate specific institutions, and should continue to do so for that purpose. 


Establishing a robust and consistent definition of job placement rates makes important strides towards 
allowing job placement rates to be verified. Table 8 at the end of this section lays out one such rate. The 
GE regulation required that all career and technical programs must establish a list of “occupations that the 
program prepares students to enter if the student completes this program of study,” as well as the SOC 
codes associated with each of these occupations.”° By correlating programs with matching codes in state 
UI databases it will become increasingly possible to provide verified job placement rates that demonstrate 
if a student is employed in occupations for which they were trained by matching the students’ occupation 
codes and programs. While just two states are currently matching occupation codes and an additional four 
states collect the SOC codes, these investments have the potential to dramatically improve the quality of 
information available to st 


tudents and policymakers. 


Additional improvements to state databases are also critical to improving job placement rates. States are 
increasingly matching information between records of students who attend colleges in the states with the 
wage information in state Ul databases. However, state database matching systems can still be dramatically 
improved including adding variables necessary for the calculation of accurate job placement rates — not 
just occupation codes — but also program information, program completion date, employment status by 
quarter, employment intensity, and occupation. States should also continue to address gaps in state data 
systems by incorporating more students, including those not attending a state’s public colleges. Similarly, 
states have the ability to calculate wage information not just on an annual but on a quarterly basis. Finally, 
states should continue to work towards limiting restrictions on the sharing of this data between states, and 
between states and the federal government. Increased funding can help in this goal. 


Experts have called the Educational Technical Assistance Act of 2002, a program that awards states grants 
of up to $20 million to build their longitudinal data systems, the “single biggest driver” in state data system 
development over the past 10 years.’7' However, the Trump Administration's Fiscal Year 2019 budget pro- 
posed eliminating the program, which is currently funded at about $32 million. Only by further investing in 
this program, is it possible to ensure that state data systems continue to develop tools to better calculate 
employment metrics including job placement rates. Investments by both the federal government and by 
individual states can substantially reduce reliance on job placement rates reported by institution and even- 
tually allow for directly matching education and workforce data to determine rates of employment. 


Finally, comparable and accurate job placement rates are of little use if they are not easily accessible. The 

GE regulation mandated the use of a standardized disclosure template to ensure that information regarding 
ob placement rates was provided online and individually in a manner that was clear and understandable to 
prospective students.’* However, ED has proposed eliminating job placement rates from the GE disclosure 
template as early as January 2019.'% While many accreditors separately require some level of disclosure of 
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job placement rates to students, ED should restore the GE job placement rate disclosures, while also im- 
proving enforcement and accessibility to ensure that consistent information is provided in an accessible and 
transparent manner. 


To improve job placement rates we recommend that the federal government take the lead in creating 
standardized, verifiable job placement rates for use by accrediting agencies, state and federal regulators for 
career education programs by: 


a. Adopting a specific, standardized robust definition of job placement for use by federal, state, and 
accrediting agencies; 


b. Building state data system capabilities to improve the calculation and verification of job placement 
rates through state data matches; and 


c. Maintaining and improving job placement rate disclosures. 
Threshold Earnings Rate 


While job placement rates must be maintained and improved by moving towards state verification, job 
placement rates are most helpful in determining the share of students succeeding in career programs. Ideal- 
ly, students and others would have access to a metric that enables them to easily make meaningful compar- 
isons of student success across all degree programs in higher education nationwide. 


The best way to calculate a threshold earnings rate for individual programs is by matching graduates’ data 
with wage information at the federal level. However, to create such a comprehensive, comparable, and 
transparent threshold earnings rate requires the policy changes detailed below. 


Perhaps most critically, in order to facilitate a more complete, efficient, and accurate calculation of nationally 
comparable program-level employment metrics, Congress should authorize the creation of a secure federal 
student-level data network (SLDN), as proposed in the College Transparency Act.’4 The HEA currently pro- 
hibits ED from developing and maintaining a comprehensive student-level postsecondary education data in- 
frastructure. A federal SLDN would substantially improve existing information, regarding the earnings rates 
of students, by including the more than 25 percent of students who are not currently captured because 
they receive no student loans or grants.’ An SLDN would also empower federal agencies to systematically 
compute nationally comparable metrics on scale without relying on institutions to report large quantities 
of data. It would allow for the easy disaggregation of data by completion status and student characteristics 
and would facilitate transparency and research, while ensuring the underlying student-level data remains 


securely protected. 


More immediately, ED should restore the threshold wage earnings rates for institutions to the College 
Scorecard, the public facing accessible platform created in 2015. This rate is detailed in Table 8. In Septem- 
ber 2018, ED removed this information from the College Scorecard (although it is still currently available 
through the College Scorecard dataset available to researchers). This information is critical to help prospec- 
tive students understand not just how much students attending a school could earn, but also how many 
students were (and were not) earning above the threshold. While important existing information might 
demonstrate that graduates were earning a median amount of $50,000 a year, a threshold earnings rate 
can provide the additional context that perhaps only 60 percent of those graduates were earning more than 


$28,000. 


While Scorecard data are valuable on an institution by institution basis, it is more valuable when 

provided for each program offered by a school. For example, the threshold earnings rates might look very 
different for a program in political science than for one in computer science, and an institutional level mea- 
sure is impacted by these differences. Although its inclusion is a stated goal of both the current and former 
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Administrations, program-level data are still not available via the College Scorecard.”° These data are criti- 


cal to decision-making, as there is substantial program-level variation in 


employment outcomes, and should 


be made available both on the public facing College Scorecard and the downloadable College Scorecard 


dataset.'?” 


Additionally, while creation of an SLDN would make the existing data sharing agreements to calculate 
threshold earnings more complete and less complicated, the federal government should also explore addi- 


tional data sharing agreements in order to enhance the information avai 
data. 


Ultimately, to improve threshold earnings rates we recommend that the 


able, including capturing state level 


federal government create and 


publish a nationally comparable, program-level threshold earnings rate through a federal data match by: 


a. Restoring important information regarding the wages earned by 
old to the public facing College Scorecard tool; 


4 


graduates above a certain thresh- 


b. Reporting College Scorecard employment metrics by program; and 


c. Authorizing a federal student-level data network to improve the 
metrics. 
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calculation of federal employment 


TABLE 8: EMPLOYMENT METRIC RECOMMENDATIONS 


Element Needing Clarification Threshold Earnings Rate’ piandetdized ob 
Placement Rate 


Scope: Should this rate be calcu- 
lated at the institution- or program 
level? What institutions or programs 
should this rate be calculated for? 


Class: Are cohorts based on starting 
classes or completing classes? If 
starting, how long after a cohort's 
start is job placement measured? 


Exclusions: Are certain subsets of 
students excluded? E.g., those unable 
to seek employment due to con- 
tinuing education, military duty, visa 
restrictions, incarceration, medical 
conditions, or death. What about 
those who are not seeking employ- 
ment? 


Breadth: Must a graduate be 
employed in an occupation direct- 
ly in the field for which they were 
trained, in a related occupation, or 
in any comparably or better paying 
occupation? 


Intensity: Is all work counted or only 
full-time, full-year work? What about 
temporary work? 


Timing: How long after program 
completion must a graduate gain em- 
ployment? Must they be employed 
for a certain duration? 


Verification: How must data be 
verified? 


Sample Size: What share of graduat- 
ing students need to be included in 
calculations? 


All institutions eligible for federal 
financial aid. 
Eventually shift to all such programs. 


Federal student aid recipients who 
started the school six years prior. 
Eventually shift to those who com- 
pleted the program one year prior 


Excludes those continuing education, 
as verified by a student-level data 
network or approximated by having 
a federal loan that is in in-school 
deferment status for at least 30 days 
during the measurement year 


Any individual with earnings ex- 
ceeding the median wage of workers 
ages 25 to 34 who self-identify 

high school as their highest level 

of education (currently $28,000). 
Earnings are defined as the sum of 
wages and deferred compensation 
from W-2 and Schedule SE forms ina 
given year. 


Any work that meets the above 
criteria. 


Six years after program entry, which 
is estimated from federal aid filings. 


Eventually shift to one year after 
program completion. 


Data must be from a federal data 
match between data on federally aid- 
ed students and earnings data from 
administrative tax records. 


All those included in the federal data 
match. 


All career education programs. 


Students who completed the program 
during the award year. 


Excludes those documented as not avail- 
able for employment due to death, disabil- 
ity, active military duty, visa restrictions, or 
continued education. 


Either (1) a position in the list of SOC job 
titles for which ED approved the program 
or (2) a position in the program's industry 
with a starting salary equal to or exceed- 
ing the 25th percentile of salaries reported 
for the highest paid SOC code approved 
for the program. 


Work must be for at least 32 hours per 
week. 


Within 180 days of completing the pro- 
gram or graduating from the program the 
completer or graduate has been employed 
for at least 13 weeks. 


If a license or certification is required or 
generally requested for positions in the 
occupation, then within 180 days after the 
results are available from the first exam 
that the completer/graduate would have 
been able to take after completing the 
program. 


Data must be from a state longitudinal 
data system if possible. Otherwise, docu- 
mentation must include written communi- 
cation from the source including who the 
source is. If information is obtained ver- 
bally, the institution must send the source 
and maintain a confirming letter to the 
provider of the information setting-forth 
in specific detail the information provided 
and the date it was provided. 


If using a state longitudinal data system, 
all those included in the state data match. 
Otherwise, at least 50 percent, and if 
between 50 and 80 percent, data must be 
assessed for nonresponse bias. 


* https:/collegescorecard.ed.gov/assets/FullDataDocumentation.pdf; https://collegescorecard.ed.gov/assets/UsingFederal Data ToMeasureAnd|ImprovePerformance.pdf 


** This rate is based off a rate proposed during GE negotiations: https://www2.ed.gov/policy/highered/reg/hearulemaking/2012/21jobplacement-rate-as-disclosure93013.pdf 
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CONCLUSION 


The United States invests over half a trillion dollars in higher education each year. Yet student outcomes 
vary immensely between institutions and programs. In such a world, it is critical that students, policymak- 
ers, and other decision-makers have access to clear, accurate, and accessible information. Employment 
metrics can be particularly meaningful measures of educational outcomes for students that, alongside other 
important metrics, can help others evaluate the relative returns of different institutions and programs. How- 


ever, the large variation in current definitions and requirements mandated by federal, state, and accrediting 
agencies has resulted in a status quo where employment rates are not comparable, accurate, or accessible. 


To address this, policymakers should align and improve job placement rates as well as create and publish 
nationally comparable, program-level threshold earning rates. Job placement rates that are consistently 
defined across programs and accurately calculated through state data matches can be a meaningful tool for 
students and others to assess career and technical programs, particularly those programs that offer eco- 
nomic advancement in low wage occupations. Threshold earnings rates that are measured by including all 
graduates and all earnings, additionally offer valuable information on the employment outcomes of grad- 
uates across all higher education degrees and programs. Together, improving job placement and threshold 
earnings rates will help pave the way to a world where students and others have access to a more robust 
system of clear and accurate information that helps empower them to make the best decisions about their 
futures, and where policymakers have more clear guidance on how to best direct scarce resources to sup- 
port students. 
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